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ABSTRACT 

The Electronic Procedures Experiment 
(EPROC) was flown as part of the 
Human Factors Assessment (HFA) 
experiment aboard the SpaceHab- 
l/STS-57 mission. EPROC is concerned 
with future, longer-duration missions 
which will increasingly rely on 
electronic procedures since they are 
more easily launched, updated inflight, 
and offer automatic or on-request 
capabilities not available with paper. 

A computer-based task simulating a 
Space Station Propulsion System task 
was completed by one crewmember. 
The crewmember performed the task 
once using paper and once using 
computer procedures. A soldering and 
desoldering task was performed by 
another crewmember. Soldering was 
completed with paper procedures and 
desoldering was completed using 
computer procedures. 


Objective data was collected during 

each task session from the computer 
programs, videotapes, and crew 
notations in the paper and computer 
procedures. After each task session, 
subjective data was collected through 
the use of a computer-based 
questionnaire program. Resultant 

recommendations will be made 
available to future designers of 

electronic procedures systems for 
manned-space missions and other 

related uses. 

INTRODUCTION 

Experiment Description 

The primary concerns of Human 
Factors engineers at NASA's Human- 
Computer Interaction Laboratory 
(HCIL) are the investigation and 
evaluation of human-machine 
interfaces unique to spaceflight which 
affect crew productivity and ultimately 
mission success. The Human Factors 
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Assessment (HFA) was an experiment 
conducted aboard SpaceHab l/STS-57 
by the HCIL. During this mission, HFA 
personnel evaluated the design and use 
of electronic procedures (EPROC). 

All Shuttle onboard tasks are currently 
performed using written paper 
procedures. This represents a large 
amount of launch weight and valuable 
stowage space. There are also 
particular problems with using paper 
procedures with hands-on tasks. For 
example, it is cumbersome for 
crewmembers working in a glovebox to 
take their hands off the task to turn a 
page of the procedures or to make an 
annotation. There are also limitations 
on the amount of information that can 
be presented in onboard paper 
procedures. Electronic, computerized 
procedures have none of these 
problems. The amount of information 
that can be made available and the 
capabilities that can be provided via 
computers to improve crewmembers' 
performance make electronic 
procedures worthy of investigation. 

The goal of the HFA-EPROC experiment 
was to determine Human Factors 
requirements for electronic 
procedures systems in flight 
environments. Performance measures 
were taken for the same task using 
both computer and paper procedures. 
Advantages and disadvantages of each 
procedure type were noted. In 
addition, several automated procedures 
capabilities were provided to the 
crewmembers for evaluation. Thus, 
the investigation could identify the 
benefits of paper and the potential 
benefits of computer presentation; 
rather than solely making a 
comparison between the two. 

The HFA-EPROC experiment consisted of 
two types of tasks: a computer task and 
a non-computer task. The computer 
task consisted of a simulated Space 
Station Propulsion System task which 
involved interacting with a graphical 
interface to configure the system. The 


task was performed once with 
computer procedures and once with 
paper procedures. This type of task 
was included because future missions 
will be commanded entirely via 
graphical software interfaces where 
crewmembers read on-screen 
procedures and then configure systems 
by clicking on icons and soft buttons. 

The non-computer task portion of the 
investigation consisted of a 
solder/desolder experiment. This 
portion was performed in conjunction 
with the SpaceHab Tools and Diagnostic 
Systems - Solder Equipment (TDS-SE) 
experiment. The solder portion was 
completed using paper Flight Data File 
(FDF) procedures, and the desolder 
portion was completed using computer 
procedures. This non-computer task 
was included to collect information on 
the use of electronic procedures with a 
hands-on glovebox task. Because of the 
hands-intensive nature of the 
glovebox task, voice input was one of 
the computer capabilities investigated. 

Previous research into paper and 
computer procedures has been 
performed in the HCIL at the NASA 
Johnson Space Center (O'Neal 1992; 
O'Neal and Manahan 1990; Desaulniers, 
Gillan, and Rudisill 1989). Results from 
this research and reviews of relevant 
literature (Johns 1988; Kelly 1988) 
provided the basis for the design of the 
HFA-EPROC experiment. 

METHOD 

Subjects 

Two crewmembers were recruited for 
the computer task and one 
crewmember was recruited for the 
soldering task. Additional subjects and 
trials were not possible due to mission 
timeline constraints. During the STS- 
57 mission, one crewmember was 
unable to participate due to unexpected 
mission difficulties; therefore only one 
crewmember participated in the 
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computer task, plus the crewmember 
in the soldering task for a total of two 
subjects. 

Apparatus and Materials 

A Macintosh Powerbook 170 was used to 
run the custom-built electronic 
procedures software. The electronic 
procedures and the computer task 
display were created with Supercard. 
The cursor control device used was a 
slightly modified version of the 
standard PowerBook trackball. 

The electronic procedures software was 
custom-built to investigate the 
usability of the interface. The display 
was split into halves vertically. The 
procedures were presented on the left- 
hand side; the crewmember scrolled 
through to complete the task. The task 
was completed on the right side of the 
display, where a simulation of the 
Space Station Freedom (SSF) core 
system Propulsion display was 
presented. The display was a direct 
manipulation interface where the user 
could click on icons representing 
system objects such as valves or 
heaters and change the parameters 
associated with those objects (see 
Figure 1). The software kept track of 
task times (between each step in the 
procedure), the sequence of window 
openings and closings, and the 
sequence of button presses. 

The non-computer task included the 
use of a voice input system (Voice 
Navigator software by Articulate 
Systems). The system was used solely to 
move from step to step in the 
procedure. 

Design 

The experiment used a simple within- 
subjects design. The independent 
variable was Procedure Type (Paper vs. 
Computer). Dependent variables were: 
total time on task, time on subsets of 


tasks, error rate, and subjective 
ratings. 

This basic design was repeated for each 
task type: Computer and Non-Computer 
(see Table 1). 



Computer 

Task 

Paper 

Procedures 

crewmember 1 


Computer 

Procedures 

crewmember 1 



Non-Computer 

Task 

Paper 

Procedures 

crewmember 2 


Computer 

Procedures 

crewmember 2 





Table 1. The experimental design 


Subjective ratings were collected via a 
computerized questionnaire that was 
presented after the completion of each 
task. The questionnaire ratings were 
anchored by using 7 point Likert 
scales. 

Procedure 

The crewmembers were trained on 
their respective tasks during formal 
familiarization, hands-on, and timeline 
training sessions. Crewmember 1 also 
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requested and completed several task 
review sessions prior to the mission. 

For both the computer and non- 
computer tasks, the procedural 
information available to the 

crewmember was identical in the 
paper and computer versions of the 
procedures. What differed were 

capabilities to access the information. 
Table 2 summarizes these differences. 


• 



HFR-EPROC Computer Task - Computer Procedures 






















Computer Procedures 


Paper Procedures 


• Immediate access to diagrams, 
schematics, and malfunction procedures 

• Immediate access to step details 

• Notes, Cautions, and Warnings 
automatically displayed only when 
relevant 

• Current step highlighted to assist in 
placekeeping 

• Placekeeping input through use of 
onscreen buttons 

• Timing information tracked 
automatically through initial input and 
use of onscreen buttons 

• Annotations and comments accepted 
through available notepad 

• Scrolling provided through onscreen 
buttons and manual use of scroll bars 

• Voice input available for increased 
hands-free procedure operation (non- 


• Diagrams, schematics, and malfunction 
procedures in an appendix 

• Step detail information in a separate 
table 

• Notes, Cautions, and Warnings printed 
along with procedure steps 

• Current step not highlighted 

• Placekeeping possible only through 
manual mark-up of procedures 

• All timing information tracked 
manually 

• Annotations and comments available 
through pre-defined blank lines or 
other markings 

• No scrolling facilities provided 

• No voice input facility provided 


computer task only) 

Table 2. Comparison of features provided with each procedure type 


Computer Task 

Crewmember 1 began each computer 
task session by setting up the computer 
in the SpaceHab compartment on 
either the workbench or a computer 
table. Setup included plugging the 
computer in, opening it up, and 
turning it on. The computer was 
attached to the surface of the table 
with Velcro. The crewmember stayed 
in place by using foot restraints. 


Crewmember 1 first completed the 
computer task session while using 
paper procedures and then completed 
another computer task session while 
using computer procedures. Figure 1 
shows the display used with computer 
procedures. Note that for paper 
procedures the left side of the display 
remained blank. The right side of the 
display remained the same for both 
tasks. 
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Non-Computer Task 

Crewmember 2 began the soldering 
sessions with the setup of the glovebox 
apparatus. While performing the 
computer procedures session, the 
Powerbook was set up and attached 
with Velcro to a locker to the 
crewmember's left in a flat upright 
position. The Voice Navigator headset 
was plugged in and the headset was 
donned. 

Crewmember 2 then performed a 
soldering task session while using the 
paper procedures. The soldering task 
consisted of soldering some pre- 
selected sites on an electronics board 
while following the procedures. Next, 
the crewmember completed a 
desoldering task session on a different 
electronics board while using the 
computer procedures. The computer 
procedures allowed the crewmember to 
advance to the next step in the 
procedures via a voice command for 
"hands-free" operation. 

Objective data was gathered for both 
computer and non-computer task 
sessions via the computer programs, 
videotapes, and FDF procedure 
annotations. This provided baseline 
data on migrating from paper to 
computers in space. After each task 
session, subjective data was gathered 
through the use of a computer-based 
questionnaire program, providing data 
on what to include and what to avoid in 
the design of future electronic 
procedures systems. 

RESULTS AND DISCUSSION 
Computer Task 

Due to a late return of flight data, a full 
data analysis has not yet been 
completed. Computer data and 
videotape data have not yet been 
analyzed; thus the results below 
include only the completion times and 


subjective comments. Data from the 
non-computer task in particular are 
insufficient for presentation at this 
time. Thus, only preliminary data from 
the computer task are presented. 

In addition to overall task completion 
times, task times were broken down 
into subsets (thirds) in order to get a 
more granular look at the 
crewmember’s ability to complete the 
task. The overall task completion time, 
as well as all of the individual subset 
completion times were faster for the 
computer procedures (see Table 3). 
Formal statistical tests are not 
appropriate here since the data 
represent only a few data points from 
one subject. However, the consistency 
in trends among each of the sets of 
completion times indicates that there 
probably is a real time advantage for 
the computer procedures. 



Paper 

Computer 

Overall 

17.42 

14.83 

Subset 1 

5.02 

3.87 

Subset 2 

2.98 

2.02 

Subset 3 

9.43 

8.23 


Table 3. Overall and task subset 
completion times in minutes. 


A full error analysis has not yet been 
completed. However, a preliminary 
look at the data indicates no significant 
errors. 

Overall, the computer procedures were 
rated very favorably in the 
questionnaire. Regarding the ease of 
use of the computer procedures 
interface, the crewmember's comment 
was "The format of the procedures was 
very user friendly and resulted in the 
task being easily performed." 
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The primary advantage of computer 
procedures over paper procedures, as 
identified by the crewmember, was 
that the current step was highlighted 
automatically. This released the 
crewmember from the burden of 
keeping their place in the procedures. 
Another comment regarding 
highlighting was "The procedures 
were very easy to read. The 
highlighting assisted tremendously in 
keeping your place in the procedures. 
This method assures a 'check and 
balance' approach to following 
through required procedures." 

One significant improvement that was 
identified and should be included in 
any future procedures interface was 
the capability for the astronauts to be 
able to move on to the next step via the 
keyboard or by trackball. The addition 
of keyboard redundancy allowed the 
crew to move on to the next step in the 
procedures while keeping the cursor 
in the working portion of the display 
(the task display). 

Ultimately, when asked which 
procedures the crewmember would 
prefer to use if they were given the 
choice between paper and computer, 
the crewmember responded with "I 
definitely preferred the computer 
procedures." 

The questionnaire data suggested some 
possible reasons for the quicker task 
times while using computer 
procedures. One comment made about 
using the paper procedures was "The 
necessity to use paper and pencil to 
follow through the procedures causes 
some overhead in zero g. The extra 
time necessary to clip or tether 
procedures in the vicinity of the work 
area and to ensure procedures and 
writing utensils are not free floating 
extends the time required to complete 
the task." Another possible reason for 
the time difference between computer 
and paper procedures could be the 
order of completion. The paper 
procedures were completed first, 


therefore the task would have been 
fresh in the mind of the crewmember 
as they completed the computer task. 
However, this effect should have been 
significantly diminished since the task 
had been rehearsed many times before 
the actual mission. Order and practice 
effects should have been minimal. 

CONCLUSIONS 

Because Shuttle missions currently use 
paper procedures, one objective was to 
establish the paper procedures 
usability data as a minimum baseline 
for performance while using computer 
procedures. Data reviewed thus far 
would indicate that computer 
procedures can be used in the future, 
in place of paper procedures, with no 
significant loss in productivity. 

After the full data analysis has been 
completed, Human Factors design 
guidelines will be created, helping 
designers create more powerful, usable 
electronic procedures systems. In the 
future, longer-duration missions will 
rely increasingly on electronic 
procedures since they are more easily 
launched, updated inflight, and offer 
automatic or on-request capabilities 
not available with paper. 

To facilitate future migration to 
electronic procedures, performance 
must at least be equal to performance 
achieved with paper procedures. This 
investigation has begun to confirm 
that electronic procedures are a 
feasible alternative and can offer 
many benefits over paper 
presentation. 
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